Speech recognition systems are typically trained with large speech data sets collected from a large number of speakers. Even when trained with a large training data set, speaker adaptation is still applied to the trained speech recognition system in order to enhance recognition performance for a particular user. Many approaches of speaker adaptation have been used in existing automatic speech recognition (ASR) systems. Existing speaker adaptation techniques usually make use of the speaker specific speech data collected by the ASR systems.